This description relates to reducing latency in performing a task among distributed systems, for example, in a network.
Such tasks can include synchronizing data stored on the distributed systems, or providing information or a service among the distributed systems, for example, between a server and a client.
An individual task may include only a single step performed at one of the distributed systems on behalf of, for the benefit of, or at the request of another one of the distributed systems. In some cases, however, a task may be complex, including many steps, such as a series of discrete requests and responses among the distributed systems. To complete a task, the distributed systems exchange communications through the network.
Generally, each step of a task entails two separate communications through the network, one that passes from one of the distributed systems to the other and contains a request, and the other of which contains the result and returns in the other direction. A task that involves more than one step will typically involve more than two communications, sometimes many.
For example, a client may first send an initial request to a server, which processes the request and sends an interim result back to the client. Sometimes, based on the nature of the interim result, the client may then send a second request, the server returns another interim result, and the back and forth continues until the client eventually receives the needed data or service, that is, until the task is completed.
There are latencies (time delays) in performing a task, including each of its steps. A variety of terminology has been used to refer to the different latencies involved. Reference is made to the explanation of latencies contained in the Wikipedia entry “Latency (engineering)” incorporated here by reference in its entirety in the form in which it appeared as of the filing of this patent application.
Here, we sometimes use the phrase “task operational latency” to refer to the entire time that elapses from the start of the task until the end of the task, including all of the processing and communication time entailed in completing every step needed to finish the task. Similarly, we call the time required to complete a step of the task “a step operational latency.”
In the example of a request made by a client to a server, the step operational latency includes the times required to prepare the request at the client and the reply at the server, and to decode the request at the server and the reply at the client, the time spent by the server to perform the requested action, and the communication latency for the communication (e.g., the request and the reply) transmitted between the client and the server to initiate and to finish that step of the task.
By communication latency, we mean, for example, the time spent to send a communication through a network, from the system that initiated the communication (e.g., the client) to the system that is to act on the contents of the communication (e.g., the server), and back again, without including any step operational latency that may occur at the server.